Apparatus and method for generating a disparity map in a receiving device

ABSTRACT

An apparatus and method for generating a disparity map in a receiving device, e.g., a set-top box, that enables user control for adjusting image depth in a stereo image set are provided. The apparatus and method provide for receiving a signal comprising a left eye image and a right eye image, encoding the left eye image as a reference image, predictively coding the right eye image using the coded left eye image as the reference image, capturing motion indicators generated during encoding of the right eye image, and generating a disparity map between the left eye image and right eye image using the motion indicators.

This application claims the benefit, under 35 U.S.C. §365 ofInternational Application PCT/US2011/024877, filed Feb. 15, 2011, whichwas published in accordance with PCT Article 21(2) on Aug. 23, 2012 inEnglish.

BACKGROUND OF THE INVENTION

Home entertainment systems, including television (TV) and media centers,are converging with the Internet and providing access to a large numberof available sources of content, such as video, movies, TV programs,music, etc. As a result, numerous challenges have emerged related to thedisplay and navigating process for that accessible content.

Broadcasters, eager to test the three dimensional (3D) display to thehome video market, want to adopt a broadcast format that is backwardcompatible with all existing high definition (HD) capable set-top boxesin the field. The broadcasters have recently chosen to begin 3D videobroadcasts with a half horizontal resolution split-screen encoding ofthe left and right stereo video views. In this scenario, as well asother split-screen video scenarios, it is the display device thatconverts from the split-screen format to a format that can be perceivedas stereo video by the viewer.

The Blu-ray Disc Association (BDA) has elected the multi-viewcompression algorithm, also known as Multiview Video Coding (MVC), tosupport efficient compression of stereo video stored on 3D enabledBlu-ray discs. The BDA has also specified that 3D video be encoded with1280×720 p60 or 1920×1080 p24 resolution and frame rate available foreach eye. Current and previous generation set-top boxes are not capableof supporting the decoding of MVC coded video, and are also not capableof supporting any other known method that would deliver a video streamof equivalent resolution and frame rate as mandated by the BDA. As aresult, broadcasters will be pushed to show an upgrade path to Blu-rayquality 3D video in the future. However, broadcasters will also beobligated to continue support of the initial group of 3D video customersusing legacy decoders and half horizontal resolution split-screen video.This obligation rules out a switch to MVC compression unless thebroadcaster is willing to pay for an equipment swap to upgrade thedecoders used by the initial 3D customers.

Further, with the advent of the delivery of 3D video to the home, theability to display the 3D content properly in a home environment. Oneparticular problem is related to the ability to adjust the depth offield of the 3D content while viewing. A user may not feel comfortablewatching the 3D content on a smaller display device and the userexperience of 3D may, as a result, not be enjoyable. Other similarproblems related to depth of field or other aspects of the 3D contentmay also be present. A user would benefit from the ability to makeadjustments to the 3D content prior to, or during, viewing. In order forthe user to perform these adjustments additional information, such asthe disparity mapping for the content, are needed. However, in mostcases, this additional information is not available or provided as partof the 3D content delivery.

Therefore, there is a need to generate information, such as thedisparity mapping, for the delivered 3D content in the home environmentin order to allow a user to adjust parameters related to the viewing ofthe 3D content, such as depth of field.

SUMMARY

According to one aspect of the present disclosure, a method is providedincluding the steps of receiving a signal comprising a left eye imageand a right eye image, encoding the left eye image as a reference image,predictively coding the right eye image using the coded left eye imageas the reference image, capturing motion indicators generated duringencoding of the right eye image, and generating a disparity map betweenthe left eye image and right eye image using the motion indicators.

According to another aspect of the present disclosure, an apparatus isprovided including a receiver that receives a signal comprising a lefteye image and a right eye image; an encoder that encodes the left eyeimage as a reference image, the encoder predictively codes the right eyeimage using the coded left eye image as the reference image and capturesmotion indicators generated during encoding of the right eye image, anda controller that generates a disparity map between the left eye imageand right eye image using the motion indicators.

BRIEF DESCRIPTION OF THE DRAWINGS

These, and other aspects, features and advantages of the presentdisclosure will be described or become apparent from the followingdetailed description of the preferred embodiments, which is to be readin connection with the accompanying drawings.

In the drawings, wherein like reference numerals denote similar elementsthroughout the views:

FIG. 1 is a block diagram of an exemplary embodiment of a system fordelivering video content in accordance with the present disclosure;

FIG. 2 is a block diagram of an exemplary embodiment of a system forthree dimensional (3D) half resolution split screen broadcasts whichprovides two dimensional (2D) legacy support in accordance with thepresent disclosure;

FIG. 3 is a flowchart of an exemplary embodiment of a process forencoding half resolution split screen broadcasts in accordance with thepresent disclosure;

FIG. 4 is a flowchart of an exemplary embodiment of a process fordecoding half resolution split screen broadcasts in accordance with thepresent disclosure;

FIG. 5 is a block diagram of an exemplary embodiment of a system forlegacy three dimensional (3D) broadcasts and full resolution 3Dbroadcasts in accordance with the present disclosure;

FIG. 6 is a flowchart of an exemplary embodiment of a process forencoding legacy three dimensional (3D) broadcasts and full resolution 3Dbroadcasts in accordance with the present disclosure;

FIG. 7 is a flowchart of an exemplary embodiment of a process fordecoding legacy three dimensional (3D) broadcasts and full resolution 3Dbroadcasts in accordance with the present disclosure;

FIG. 8 is an exemplary embodiment of a receiving device for generating adense disparity map in accordance with the present disclosure; and

FIG. 9 is flowchart of an exemplary embodiment of a process forgenerating a dense disparity map in accordance with the presentdisclosure.

It should be understood that the drawings are for purposes ofillustrating the concepts of the disclosure and is not necessarily theonly possible configuration for illustrating the disclosure.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

It should be understood that the elements shown in the figures may beimplemented in various forms of hardware, software or combinationsthereof. Preferably, these elements are implemented in a combination ofhardware and software on one or more appropriately programmedgeneral-purpose devices, which may include a processor, memory andinput/output interfaces.

Herein, the phrase “coupled” is defined to mean directly connected to orindirectly connected with through one or more intermediate components.Such intermediate components may include both hardware and softwarebased components.

The present description illustrates the principles of the presentdisclosure. It will thus be appreciated that those skilled in the artwill be able to devise various arrangements that, although notexplicitly described or shown herein, embody the principles of thedisclosure and are included within its spirit and scope.

All examples and conditional language recited herein are intended foreducational purposes to aid the reader in understanding the principlesof the disclosure and the concepts contributed by the inventor tofurthering the art, and are to be construed as being without limitationto such specifically recited examples and conditions.

Moreover, all statements herein reciting principles, aspects, andembodiments of the disclosure, as well as specific examples thereof, areintended to encompass both structural and functional equivalentsthereof. Additionally, it is intended that such equivalents include bothcurrently known equivalents as well as equivalents developed in thefuture, i.e., any elements developed that perform the same function,regardless of structure.

Thus, for example, it will be appreciated by those skilled in the artthat the block diagrams presented herein represent conceptual views ofillustrative circuitry embodying the principles of the disclosure.Similarly, it will be appreciated that any flow charts, flow diagrams,state transition diagrams, pseudocode, and the like represent variousprocesses which may be substantially represented in computer readablemedia and so executed by a computer or processor, whether or not suchcomputer or processor is explicitly shown.

The functions of the various elements shown in the figures may beprovided through the use of dedicated hardware as well as hardwarecapable of executing software in association with appropriate software.When provided by a processor, the functions may be provided by a singlededicated processor, by a single shared processor, or by a plurality ofindividual processors, some of which may be shared. Moreover, explicituse of the term “processor” or “controller” should not be construed torefer exclusively to hardware capable of executing software, and mayimplicitly include, without limitation, digital signal processor (“DSP”)hardware, read only memory (“ROM”) for storing software, random accessmemory (“RAM”), and nonvolatile storage.

Other hardware, conventional and/or custom, may also be included.Similarly, any switches shown in the figures are conceptual only. Theirfunction may be carried out through the operation of program logic,through dedicated logic, through the interaction of program control anddedicated logic, or even manually, the particular technique beingselectable by the implementer as more specifically understood from thecontext.

In the claims hereof, any element expressed as a means for performing aspecified function is intended to encompass any way of performing thatfunction including, for example, a) a combination of circuit elementsthat performs that function or b) software in any form, including,therefore, firmware, microcode or the like, combined with appropriatecircuitry for executing that software to perform the function. Thedisclosure as defined by such claims resides in the fact that thefunctionalities provided by the various recited means are combined andbrought together in the manner which the claims call for. It is thusregarded that any means that can provide those functionalities areequivalent to those shown herein.

The present disclosure relates to systems and methods for permitting thebroadcast and reception of three dimensional (3D) video content alongwith two dimensional (2D) content to service several different receptionconditions, including receiving and decoding only a 2-D image, receivingand decoding only a single video frame containing both left and rightview images at a reduced resolution, and receiving and decoding bothsignals together in order to display an improved resolution 3-D image.The systems and methods indicate transmitting, for instance, the leftview image at full resolution along with a second signal having both theleft and right view images combined at a reduced (i.e. half) resolution.The receiver may decode one or the other signal and may alternativelydecode both if it has the capability. If the receiver decodes both, itmay also sample rate upconvert, for instance, the reduced resolutionright eye view to match the image size of the decoded full resolutionleft view signal.

Initially, systems for delivering various types of video content to auser will be described. Turning now to FIG. 1, a block diagram of anembodiment of a system 100 for delivering content to a home or end useris shown. The content originates from a content source 102, such as amovie studio or production house. The content may be supplied in atleast one of two forms. One form may be a broadcast form of content. Thebroadcast content is provided to the broadcast affiliate manager 104,which is typically a national broadcast service, such as the AmericanBroadcasting Company (ABC), National Broadcasting Company (NBC),Columbia Broadcasting System (CBS), etc. The broadcast affiliate managermay collect and store the content, and may schedule delivery of thecontent over a deliver network, shown as delivery network 1 (106).Delivery network 1 (106) may include satellite link transmission from anational center to one or more regional or local centers. Deliverynetwork 1 (106) may also include local content delivery using localdelivery systems such as over the air broadcast, satellite broadcast, orcable broadcast. The locally delivered content is provided to areceiving device 108 in a user's home, where the content willsubsequently be searched by the user. It is to be appreciated that thereceiving device 108 can take many forms and may be embodied as a settop box/digital video recorder (DVR), a gateway, a modem, etc.

A second form of content is referred to as special content. Specialcontent may include content delivered as premium viewing, pay-per-view,or other content otherwise not provided to the broadcast affiliatemanager, e.g., movies, video games or other video elements. In manycases, the special content may be content requested by the user. Thespecial content may be delivered to a content manager 110. The contentmanager 110 may be a service provider, such as an Internet website,affiliated, for instance, with a content provider, broadcast service, ordelivery network service. The content manager 110 may also incorporateInternet content into the delivery system. The content manager 110 maydeliver the content to the user's receiving device 108 over a separatedelivery network, delivery network 2 (112). Delivery network 2 (112) mayinclude high-speed broadband Internet type communications systems. It isimportant to note that the content from the broadcast affiliate manager104 may also be delivered using all or parts of delivery network 2 (112)and content from the content manager 110 may be delivered using all orparts of Delivery network 1 (106). In addition, the user may also obtaincontent directly from the Internet via delivery network 2 (112) withoutnecessarily having the content managed by the content manager 110.

Several adaptations for utilizing the separately delivered content maybe possible. In one possible approach, the special content is providedas an augmentation to the broadcast content, such as providingalternative displays, purchase and merchandising options, enhancementmaterial, and similar items. In another embodiment, the special contentmay completely replace some programming content provided as broadcastcontent. Finally, the special content may be completely separate fromthe broadcast content, and may simply be a media alternative that theuser may choose to utilize. For instance, the special content may be alibrary of movies that are not yet available as broadcast content.

Referring to FIG. 2, a block diagram of an embodiment of a system 200for three dimensional (3D) half resolution split screen broadcasts whichalso provides two dimensional (2D) legacy support is shown. System 200includes a transmitter 202 for encoding and broadcasting video contentand a receiver 204 for receiving the video content, decoding the videocontent and providing the decoded video content to a display device.Transmitter 202 may typically be included with equipment located at asignal transmission center, such as broadcast affiliate manager 104described in FIG. 1. The signal produced by transmitter 202 istransmitted over a broadcast network, such as delivery network 1 (106)to receiver 204. Receiver 204 may be a set top box, gateway, computer,or other network interface device that is typically located at or near auser's home. Receiver 204 may also be included in a mobile device, suchas a personal digital assistant, tablet, or cellular telephone.

The transmitter 202 includes a source for, or otherwise acquires, a fullresolution, right view image 206 and a full resolution left view image208 of a stereo image pair. The right view image 206 and left view image208 may be provided from a content source, such as content source 102described in FIG. 1. Each of the right view image 206 and left viewimage 208 are passed to respective sample rate converters 210, 212.Sample rate converters 210, 212 convert the horizontal sample rate ofeach image to one half the original horizontal size. The one half sizeright view image 214 and one half size left view 216 are merged into asingle image in an image processor 218.

The merged image from image processor 218 is then transmitted to encoder220 where the merged image is encoded in accordance with the MotionPicture Entertainment Groups (MPEG) H.264/MPEG-4 Part 10 standard, theAdvanced Video Coding (AVC) standard, or similar compression algorithm,to create first bit stream 222. Additionally, the acquired fullresolution left view image 208 is provided to encoder 224 where the leftview image 208 is encoded in accordance with H.264/MPEG-4 Part 10, AVC,or similar compression algorithm, to create a second bit stream 226. Itis important to note that the two streams may not use the identicalcompression algorithms. For instance, encoder 220 may encode the firstbit stream 222 using a higher or lower compression rate H.264 algorithmthan encoder 224 uses for the second bit stream 226. Additionally,although the full resolution image 208 is identified as a left viewimage, its description is for convention purposes only. The fullresolution image 208 may, instead be identified as aright view image. Asa result, it is understood that the descriptions for left view image andright view image signal throughout this disclosure may be reversed.

The first and second bit streams 222, 226 from encoder 238 and encoder244 are provided to output signal processor 228. Controller 230 createsof determines identification information and provides this informationalong with any other control information to output signal processor 228.In one embodiment, controller 230 sets a packet ID for each bit stream.Output signal processor 228 merges the first and second bit streams 222,226 into a single bit stream 232 for transmission as part of a signal tothe receiving device 204 based on the packet ID information and thecontrol information provided by controller 230. Controller 230 alsoappends additional information to the single bit stream 232 including,but not limited to, an identification bit, an identification byte, oridentification packet. Furthermore, the controller 230 may create aprogram guide that is also transmitted as part of the signal to thereceiving device using the identification. The inclusion of programguide information allows the receiver 204 to identify the programcontent that the bit stream 232 contains.

The receiver 204 processes the incoming signal including bit steam 232,and provides a separation of the content based on the program guide. Thereceiver 204 may include a storage device 237, such as a hard drive oroptical disk drive, for recording and playing back audio and videocontent. The processed content is provided to a display device, such asdisplay device 114 described in FIG. 1. The display device may be aconventional 2-D type display or may alternatively be an advanced 3-Ddisplay. The receiving device 204 may also be incorporated into othersystems including the display device itself. In either case, severalcomponents necessary for complete operation of the system are not shownin the interest of conciseness, as they are well known to those skilledin the art.

In the receiving device 204 a signal including content, such as bitstream 232, is received in an input signal receiver 234. The inputsignal receiver 234 may be one of several known receiver circuits usedfor receiving, demodulation, and decoding signals provided over one ofthe several possible networks including over the air, cable, satellite,Ethernet, fiber and phone line networks. The desired input signal may beselected and retrieved in the input signal receiver 234 based on userinput provided through a control interface (not shown). The outputsignal from the input signal receiver 234 is provided to an input streamprocessor 236. The input stream processor 236 performs the signalselection and processing, and determines which of the first and secondbit streams are to be sent to the proper decoder. The input streamprocessor 236 will distinguish the first and second bit streams based onthe program guide sent by device 202 and the packet ID or otheridentification information for each bit stream (e.g., bit stream 222 andbit stream 226).

Additionally, the input stream processor 236 may send the received bitstreams to storage device 237. The storage device 237 allows laterretrieval and playback of the content under the control of a controller254 and also based on commands, e.g., navigation instructions such asfast-forward (FF) and rewind (Rew), received from a user interface (notshown). The storage device 237 may be a hard disk drive, one or morelarge capacity integrated electronic memories, such as static randomaccess memory, or dynamic random access memory, or may be aninterchangeable optical disk storage system such as a compact disk driveor digital video disk drive.

As described, the input stream processor 236 will separate the bitstreams and forward one of the first and second bit streams to theappropriate decoder, either decoder 238 or decoder 244. In oneembodiment, if the input stream processor 236 determines that the bitstream includes the merged image, the bit stream will be sent to decoder238. Decoder 238 decodes the merged image in accordance with one or morevideo compression decoding algorithms, such as H.264/MPEG-4 Part 10 orAVC, to a merged image 240 having a half resolution left and right view.The 3D half resolution output 242 is provided to selector 250.Controller 254 is used to determine and control which of the bit streamsis provided through selector 250 to audio/video interface 252. Theoperation of the selector 250 and controller 254 will be describedbelow.

If the input stream processor 236 determines that the bit streamincludes the full resolution left view image 208, the bit stream will besent to decoder 244. Decoder 244 decodes the left view image 208 inaccordance with one or more compression decoding algorithms to generatea left view image 246. The left view image is then outputted as a 2Dfull resolution output signal 248. The 2D full resolution output signal248 is also provided to the selector 250.

Controller 254 determines a type of a display device coupled to thereceiving device 204 via an audio/video interface 252. The audio/videointerface 252 may be an analog signal interface such as red-green-blue(RGB) or may be a digital interface such as high definition multimediainterface (HDMI). The controller 254 communicates with the audio/videointerface 252 and receives information from the audio/video interface252 as to whether a 2D or 3D display device is connected thereto. Forexample, in the case of HDMI, the controller 254 determines thecapabilities of the display device by communicating through the HDMIinterface with the display device. However, this is generally notpossible with an analog audio/video (A/V) interface. When connected byan analog AV interface, the display device setup is normallyaccomplished with buttons on the receiving device 204 which can be readby the controller, or with user interface screens where a user inputsthe display type with a remote control driven selection.

Based on the type of the display device, the controller 254 controls theselector 250 to output the appropriate output signal, either the 3D halfresolution video signal or 2D full resolution video signal. For example,if the controller 254 determines a 3D display device is coupled to theaudio/video interface 252, the 3D half resolution output 242 will besent to the audio/video interface 252 via the selector 250. If thecontroller 254 determines a 2D display device is coupled to theaudio/video interface 252, the 2D full resolution output 248 will besent to the audio/video interface 252 via the selector 250. It is to beappreciated that the above processing may originate from the signalsreceived at the input signal receiver or from content retrieved from thestorage device 237.

It is further to be appreciated that controller 230 and controller 254may be interconnected via a bus to several of the components containedwithin transmitter 202 and set-top box 204 respectively. For example,controller 254 may manage the conversion process for converting theinput stream signal into a signal for storage on the storage device 237or for display on a display device, not shown. The controller 254 alsomanages the retrieval and playback of stored content. The controller 230and controller 254 may further be coupled to a control memory, notshown, (e.g., volatile or non-volatile memory, including random accessmemory, static RAM, dynamic RAM, read only memory, programmable ROM,flash memory, EPROM, EEPROM, etc.) for storing information andinstruction code for controller 230 and controller 254. Further, theimplementation of the memory may include several possible embodiments,such as a single memory device or, alternatively, more than one memorycircuit connected together to form a shared or common memory. Stillfurther, the memory may be included with other circuitry, such asportions of bus communications circuitry, in a larger circuit.

Turning to FIG. 3, a flowchart of an embodiment of a process 300 forencoding half resolution split screen broadcasts is shown. Process 300will primarily be described with respect to the transmitter 202described in FIG. 2 but may similarly be included in the equipment foundin broadcast affiliate manager 104 described in FIG. 1. At step 302, afull resolution, right view image is acquired. Similarly, at step 304 afull resolution left view image is acquired. The right view and leftview image form a stereo image pair and may be provided from a contentsource, such as content source 102 described in FIG. 1. Additionally oneof the two images may be generated by a transmitter device, such astransmitter 202, using a 2D image and 2D-3D processing techniques. Atstep 306, each of the right view image and left view image are convertedby changing the horizontal sample rate of each image to one half theoriginal horizontal size. Next, at step 308, the one half size rightview image and one half size left view are merged into a single image.It is important to note that each one half size image typically occupiesone half of the full horizontal width of the image signal.Alternatively, each half size image may be interspersed in a patternacross the entire image signal, such as in a checkerboard pattern.

At step 310, the merged image is then encoded using a video compressionencoding algorithm to create a first encoded bit stream. In oneembodiment, the merged image may be encoded in accordance withH.264/MPEG-4 Part 10, AVC, or some similar compression algorithm. Atstep 312, the full resolution left view image from step 304 is encodedusing a video compression encoding algorithm, similar to that describedin step 310, to create a second encoded bit stream.

Next at step 314, information about the first bit stream and second bitstream is retrieved and processed to form one or segments (e.g., bits,bytes, packets) of program information. Also in step 314, the first bitstream and second bit stream are merged to form a single signal or bitstream and the information for the first bit stream and second bitstream is appended to the single bit stream. In one embodiment theinformation is appended as a program identification (PID). The PID forthe single bit stream may also be combined with PIDs from other bitstreams to form a separate program guide bit stream. Last, at step 316,an output signal containing the single bit stream is transmitted. Theoutput signal may be transmitted as a transmission signal over adelivery network, such as delivery network 106 described in FIG. 1. Thetransmission step 316 may also include additional signal processingnecessary to transmit the signal, such as additional error correctionencoding, modulation coding, digital to analog conversion, filtering,and upconverting of the analog signal. The output signal may containother bit streams as well as additional information, such as a programguide stream. It is important to note that the program guide informationmay be appended to the single bit stream instead of being created as aseparate bit stream.

Turning to FIG. 4, a flowchart of an exemplary process 400 for decodinghalf resolution split screen broadcasts is shown. Process 400 will beprimarily described with respect to receiver 204 described in FIG. 2.Process 400 may also be used as part of the operation of a receivingdevice, such as receiving device 108 described in FIG. 1. At step 402, asignal, containing the desired bit stream (e.g., bit stream 232) as wellas other content, is received from a transmission network. The signaland the content may be provided by a network service provider, such asbroadcast affiliate manager 104 described in FIG. 1, and received in aninput signal receiver, such as receiver 234. The content, including thebit stream, such as desired bit stream 232, may also be provided from astorage device, such as storage device 237 or other media device, suchas digital versatile disc (DVD) or other media.

At step 404, the received input signal is separated into multiple bitstreams. As necessary, each of first and second bit streams from thedesired bit stream 232 is provided to the appropriate decoder. Step 404may include a determination as to whether the bit stream includes themerged image. If the bit stream includes the merged image, the bitstream will be decoded in accordance with one or more video compressiondecoding algorithms, such as H.264/MPEG-4 Part 10, AVC, or other similarcompression decoding process. The decoding at step 404 produces a mergedimage having a half resolution left and right view. At step 406, asimilar decoding occurs if one of the separated bit streams includes thefull resolution left view image. The decoding at step 406 may use avideo compression decoding algorithm similar to that described in step404.

At step 408, the type of display device used for displaying the videocontent is determined. In one embodiment, the display device is coupledto the set top box 204 via an audio/video interface, such as audio/videointerface 252. The audio/video interface 252 may be an analog signalinterface such as red-green-blue (RGB) or may be a digital interfacesuch as high definition multimedia interface (HDMI). The display devicemay also be integrated with the receiving and processing components inset top 204. The determination at step 408 may be performedautomatically through a display device identification process, or may beuser selected.

Based on the type of the display determined in step 408, then, at step410 the appropriate output signal is provided to the display device. Inone embodiment, steps 408 and 410 are performed by controller 254 andselector 250. For example, if the controller 254 determines a 3D displaydevice is coupled to the audio/video interface 252, the 3D halfresolution output 242 will be sent to the audio/video interface 252 viathe selector 250. If the controller 254 determines a 2D display deviceis coupled to the audio/video interface 252, the 2D full resolutionoutput 248 will be sent to the audio/video interface 252 via theselector 250. Alternatively, steps 408 and 410 may be performed usingother elements, such as the audio/video interface 22 or decoders 238 and244. It is also to be appreciated that the above processing mayoriginate from the signals received at the input signal receiver or fromcontent retrieved from the storage device 237.

Turning now to FIG. 5, a block diagram of an embodiment of a system forlegacy three dimensional (3D) broadcasts, legacy 2D broadcasts and fullresolution 3D broadcasts is shown. System 500 includes a transmitter 502for encoding and broadcasting video content and a receiving device 504,e.g., a set-top box, gateway, or other network interface device, forreceiving the video content, decoding the video content and providingthe decoded video content to a display device. Receiving device 504 mayalso be included in a mobile device, such as a personal digitalassistant, tablet, or cellular telephone. Except as described below,both transmitter 502 and receiving device 504 operate and interface witheach other in a manner similar to system 200 described in FIG. 2.

Transmitter 502 acquires a full resolution right view image and a fullresolution left view image of a stereo image pair. The full resolutionright view image and left view image may be provided from a contentsource or in some other manner as described above. The right view imageand left view image are merged into a single image in merging circuit511 and provided to a sample rate converter 510. The sample rateconverter 510 converts the horizontal sample rate of the merged image toone half the original horizontal size.

The output of the sample rate converter 510 provides a converted image518 to encoder 520. Encoder 520 encodes the image using one or morevideo compression algorithms, such as those described above.Additionally, the acquired full resolution left view image is providedto encoder 524, and encoded using one or more video compressionalgorithms as described above. The output of encoder 520 and the outputof encoder 524, identified as first bit stream 522 and second bit stream526 respectively, are provided to output signal processor 528. Output ofsignal processor 528 operates under the control of controller 530.Controller 530 sets a packet ID, or other stream identification, foreach bit stream and the output signal processor 528 merges the first bitstream 522 and second bit stream 526 into a single bit stream 532, alongwith the packet ID or other stream identification. Output signalprocessor 528 provides an output signal, including bit stream 532, thatis transmitted from transmitter 502 to receiving device 504.Additionally, controller 530 creates a program guide that is alsotransmitted to the receiving device 504. The program guide may includethe packet ID or other information about output stream 532, as well asinformation about other bit streams transmitted to receiving device 504.The program guide is used to inform the receiving device 504 what bitstream 532, as well as any other bit streams included and transmitted aspart of the output signal, contains. The program guide may betransmitted as a separate bit stream or may be appended to output bitstream 532.

The receiving device 504 processes the bit steam 532, and provides aseparation of the content based on the program guide. The receivingdevice 504 may include a storage device, such as a hard drive or opticaldisk drive, for recording and playing back audio and video content. Theprocessed content is provided to a display device, such as displaydevice 114 described in FIG. 1. The display device may be a conventional2-D type display or may alternatively be an advanced 3-D display. Thereceiving device 504 may also be incorporated into other systemsincluding the display device itself. In either case, several componentsnecessary for complete operation of the system are not shown in theinterest of conciseness, as they are well known to those skilled in theart.

In the receiving device 504, the received signal, including content suchas bit stream 532, is received in an input signal receiver 534. Theinput signal receiver 534 may be one of several known receiver circuitsused for receiving, demodulation, and decoding signals provided over oneof the several possible networks including over the air, cable,satellite, Ethernet, fiber and phone line networks. The desired inputsignal may be selected and retrieved in the input signal receiver 534based on user input provided through a control interface (not shown).The output signal from the input signal receiver 534 is provided to aninput stream processor 536. The input stream processor 536 performs thesignal selection and processing, and determines which of the first andsecond bit streams are to be sent to the proper decoder, either decoder538 or decoder 544. The input stream processor 536 identifies anddistinguishes the first and second bit streams based on the programguide information sent by device 502 and the identification of thepacket ID, or other stream identification, for a received bit stream,such as bit stream 532. Additionally, the input stream processor 536 maysend the received bit streams, if necessary, to a storage device 537, asdescribed above.

As described above, the input stream processor 536 will separate the bitstreams and, if identified or distinguished, provide one of the firstand second bit streams to the appropriate decoder, either decider 538 ordecoder 544. In one embodiment, if the input stream processor 536determines that the bit stream includes the merged image bit stream(e.g., bit stream 522), then the merged image bit stream will be sent todecoder 538. Decoder 538 decodes the merged image bit stream using avideo compression algorithm in accordance with H.264/MPEG-4 Part 10, AVC(Advanced Video Coding), or other algorithm. Decoder 538 produces a 3Dhalf resolution output image signal 542 that is a merged image 540having a half resolution left and right view. The 3D half resolutionoutput 542 is provided to selector 550 which is under control ofcontroller 554. The operation of the selector 550 and controller 554 aresimilar to the selector and controller described above in relation toFIG. 2.

If the input stream processor 536 determines, identifies, ordistinguishes that the bit stream including the full resolution leftview image (e.g., bit stream 526) is provided as part of the receivedsignal, this bit stream is provided to decoder 544. In decoder 544, theleft view image is decoded in accordance with H.264/MPEG-4 Part 10, AVC,or other decoding process to produce a 2D full resolution output signal548 that is a left view image 546. The 2D full resolution output signal548 from decoder 544, like the output of decoder 538, is also sent tothe selector 550, which is under control of the controller 554.

The half resolution right view portion of the 3D half resolution outputsignal 542 is also provided to sample rate converter 556. Sample rateconverter 556 upconverts the horizontal sample rate of the right viewportion of the image to back to full resolution. Alternatively, theentire 3D half resolution output signal 542 may be provided to samplerate converter 556. Sample rated converter 556 may operated to discardthe left view portion of the signal before upconverting the right viewportion. The upsampled right view image may be a 1080 i horizontal splitimage or 720 p vertical split image. The upconverted right view imagesignal is provided along with the 2D full resolution output signal 548from decoder 544 to the merging circuit 557. Merging circuit 557combines the upconverted right view image with the full resolution leftview image to produce left and right full resolution output signal 558.The full resolution output signal is also provided to selector 550,which is under the control of controller 554.

It is important to note that, based on the upconversion performed insample rate converter 556 on the right view portion of the 3D halfresolution output signal 542, some form of sample rate conversion mayalso be performed on the 2D full resolution output signal 548. Forexample, the 2D full resolution output signal 548 may be 1080 i format,while the upconverted right portion signal may be 720 P. In order tomatch the images, a sample rate conversion of the 2D full resolutionoutput signal 548 from 1080 i to 720 P may be necessary. The sample rateconversion may be performed in decoder 544, or may be performed in aseparate sample rate converter, not shown.

Although, an additional sample rate conversion of the left view imageportion of 3D half resolution output signal 542 may also be performedalong with the right view image portion in sample rate converter 556,the use of the 2D full resolution output signal 548 for the left viewimage results in a higher visual quality image. The process of samplerate conversion may be produce noticeable artifacts, such as imageerrors or distortions. By using, the 2D full resolution output signal548, the viewer may be less aware of the sample rate conversionartifacts because these artifacts may only be present in the right eyeview.

Controller 554 determines a type of a display device coupled to thereceiving device 504 via an audio/video interface 552. The audio/videointerface 552 may be an analog signal interface such as red-green-blue(RGB) or may be a digital interface such as high definition multimediainterface (HDMI). The controller 554 communicates with the audio/videointerface 552 and receives information from the audio/video interface552 as to whether a 2D display device, 3D legacy display device or 3Dfull resolution display device is connected thereto. Based on the typeof the display device, the controller 554 controls the selector 550 tooutput the appropriate output signal. For example, if the controller 554determines a legacy 3D display device is coupled to the audio/videointerface 552, the 3D half resolution output 542 will be sent to theaudio/video interface 552 via the selector 550. If the controller 554determines a 2D display device is coupled to the audio/video interface552, the 2D full resolution output 548 will be sent to the audio/videointerface 552 via the selector 550. If the controller 554 determines afull resolution 3D display device is coupled to the audio/videointerface 552, the 3D full resolution output 558 will be sent to theaudio/video interface 552 via the selector 550. It is to be appreciatedthat the above processing may originate from the signals received at theinput signal receiver or from content retrieved from the storage device537.

Turning now to FIG. 6, a flowchart of an exemplary process 600 forencoding legacy 3D broadcasts, legacy 2D broadcasts and full resolution3D broadcasts is shown. Process 600 will primarily be described withrespect to the transmitter 502 described in FIG. 5 but may similarly bedescribed with respect to transmitter 202 described in FIG. 2 or may beincluded in the equipment found in broadcast affiliate manager 104described in FIG. 1. At step 602, a full resolution, right view image isacquired. Similarly, at step 604 a full resolution left view image isacquired. The right view and left view image form a stereo image pairand may be provided from a content source, such as content source 102described in FIG. 1. Additionally one of the two images may be generatedby a transmitter device, such as transmitter 502, using a 2D image and2D-3D processing techniques. At step 606, each of the right view imageand left view image are converted by changing the horizontal sample rateof each image to one half the original horizontal size. Next, at step608, the one half size right view image and one half size left view aremerged into a single image. It is important to note that each one halfsize image typically occupies one half of the full horizontal width ofthe image signal. Alternatively, each half size image may beinterspersed in a pattern across the entire image signal, such as in acheckerboard pattern.

It is important to note that depending on the display format for theoriginally received image signals at step 502 and step 504, it may beadvantageous adjust the sample rate in order to re-format the resultantsingle image at the highest possible display format. For example, if theoriginal left view image and right view image were in a 720 p format,then the sample rate conversion at step 606 and merging at step 608should be performed to create a merged single image that is in a 1080 pformat. As a result, the highest possible video quality for the mergedimage signal is maintained and transmitted for use by a receiver, suchas receiving device 504.

At step 610, the merged image is then encoded using a video compressionencoding algorithm to create a first encoded bit stream. In oneembodiment, the merged image may be encoded in accordance withH.264/MPEG-4 Part 10, AVC, or some similar compression algorithm. Atstep 612, the full resolution left view image from step 604 is encodedusing a video compression encoding algorithm, similar to that describedin step 610, to create a second encoded bit stream.

Next at step 614, information about the first bit stream and second bitstream is retrieved and processed to form one or segments (e.g., bits,bytes, packets) of program information. Also in step 614, the first bitstream and second bit stream are merged to form a single signal or bitstream and the information for the first bit stream and second bitstream is appended to the single bit stream. In one embodiment theinformation is appended as a program identification (PID). The PID forthe single bit stream may also be combined with PIDs from other bitstreams to form a separate program guide bit stream. Finally, at step616, an output signal containing the single bit stream is transmitted.The output signal may be transmitted as a transmission signal over adelivery network, such as delivery network 106 described in FIG. 1. Thetransmission step 616 may also include additional signal processingnecessary to transmit the signal, as described earlier. The outputsignal may contain other bit streams as well as additional information,such as a program guide stream. It is important to note that the programguide information may be appended to the single bit stream instead ofbeing created as a separate bit stream.

Turning now to FIG. 7, a flowchart of an exemplary process 700 fordecoding legacy 3D broadcasts, legacy 2D broadcasts and full resolution3D broadcasts is shown. Process 700 will be primarily described withrespect to receiving device 504 described in FIG. 5 but may similarly bedescribed with respect to receiver 202 described in FIG. 2. Process 700may also be used as part of the operation of a receiving device, such asreceiving device 108 described in FIG. 1. At step 702, a signal,containing desired video content in a bit stream, as well as othercontent, is received from a transmission network. The signal and thecontent may be provided by a network service provider, such as broadcastaffiliate manager 104 described in FIG. 1, and received in an inputsignal receiver, such as receiver 534. The content, including the bitstream, may also be provided from a storage device, such as storagedevice 237 or other media device, such as digital versatile disc (DVD)or other media.

At step 703, the received input signal is separated into multiple bitstreams. The separation step 703 includes identification of theindividual bit streams, such as a determination and identification of afirst bit stream as a merged half resolution 3D video image andidentifying a second bit stream as a full resolution 2D image as a leftview image. The separation step 703 also provides each of the identifiedfirst and second bit streams from the desired bit stream to theappropriate decoder. Based on the determination and identification instep 703, at step 704, the first bit stream, a half resolution 3D videosignal, is decoded in accordance with one or more video compressiondecoding algorithms, such as H.264/MPEG-4 Part 10, AVC, or other similarcompression decoding process. The half resolution 3D signal may be inseveral formats, including a split screen 3D video signal or acheckerboard video signal. The decoding at step 704 produces a mergedimage having a half resolution left and right view. At step 706, asimilar decoding occurs for the second bit stream, a 2D full resolutionimage signal. The decoding at step 706 may use a video compressiondecoding algorithm similar to that described in step 704.

At step 708, the right view portion of the half resolution merged imagesignal is sample rate converted to produce a full size and fullresolution right view image signal. Although the right view image signalis a full resolution signal, it is understood that the sample rateconversion will introduce image artifacts to the full resolution rightview image. At step 710, the decoded and converted full resolution rightview image generated in step 708 and the decoded full resolution leftview image are merged into a single full resolution 3D output signal. Itis important to note that the single full resolution 3D output signalfrom step 710, the half resolution 3D output signal from step 704, andthe full resolution 2D (left view) output signal from step 706 areavailable and provided for display.

Next, at step 712, the type of display device used for displaying one ofthe output signals generated above is determined. In one embodiment, thedisplay device is coupled to the receiving device 504 via an audio/videointerface, such as audio/video interface 552. The determination at step712 may be performed automatically through a display deviceidentification process, or may be user selected. In one embodiment, if a2D display device is used, the full resolution 2D output signal isselected. In a 3D display device is used, either the half resolution 3Dresolution output signal or the full resolution 3D output signal isselected. The choice of a half resolution or a full resolution 3D outputsignal may be made by a user through a user interface selection. Inanother embodiment, the full resolution 3D output signal is alwaysselected if it available, as determined in step 703.

Based on the type of the display determined in step 712, then, at step714 the selected output signal, either a 2D full resolution left viewimage signal, a half resolution 3D left view and right view imagesignal, or a full resolution 3D left view and right view image signal,is provided to the display device.

It is important to note that although system 500, process 600, andprocess 700 are described as operating on a half resolution 3D left viewand right view image signal, other partial, or reduced, resolution 3Dsignals may also be used. For instance, 3D signal including a reducedresolution (other than half) left view image and a reduced resolution(other than half) right view image may be used, with the resolution ofthe left view image being different from the resolution of the rightview image.

Certain steps in process700 may be modified or omitted based on aspecific implementation. For instance, the display determination step712 may be performed prior to the decoding steps 704 and 706. Based onthe type of display device used, one or the other of decoding steps maybe omitted or disabled.

A system and method for legacy three dimensional (3D) broadcasts, legacy2D broadcasts and full resolution 3D broadcasts has been described. Thereceiving device 504 will decode the legacy 2D broadcast for the lefteye view and the legacy 3D split screen broadcast. The full resolutionright view is created by scaling the right half of legacy 3D splitscreen broadcast. In this manner, full resolution 3D may be providedwith no new compression techniques required.

As described above, the embodiments in FIGS. 2-7 relate to receiving andprocessing the bit streams used for 3D broadcasts includes receiving adata signal that includes two different bit streams merged into the datasignal. The first video signal represents a two-dimensional image at afirst video resolution and the second video signal represents athree-dimensional image at a second resolution. The embodiments alsoinclude decoding the first video signal if an output display type is atwo-dimensional display type, decoding the second video signal if theoutput display type is a first three-dimensional display type, anddecoding the first video signal and second video signal simultaneouslyif the output display type is a second three-dimensional display type.

The embodiments in FIGS. 2-7 also relate to processing a received signalto provide a 3D video output. The embodiments include receiving asignal, the signal including a first video signal representing atwo-dimensional image at a first video resolution and a second videosignal representing a three-dimensional image at a second resolution,determining a type of display device for viewing video content, decodingthe first video signal, decoding the second video signal, and providingan output signal to the display device, the output signal including acombination of the first decoded video signal and a portion of thesecond decoded video signal if the type of display device is a threedimensional display device.

In another embodiment, a system and method for user adjustable disparitymapping in a receiving device is provided. The system and method uses anH.264 encoder in the receiving device to assist in generating a densedisparity mapping of a received and decoded pair of stereo images, i.e.,a right view image and a left view image. The two images are alignedconsecutively in time and passed through the encoder to generateencoding information such as motion vectors. The resulting motioninformation is used to generate a disparity map that can be used as auser control for adjusting the image depth in the stereo image set. Sucha system is useful if the signal transmission includes sendingstereoscopic images but omits transmission of a depth map.

Turning now to FIG. 8, an exemplary embodiment of a receiving device 800for generating a dense disparity map is shown. It is to be appreciatedthat, except as described below, elements similar to those describedabove in relation to FIGS. 2 and 5 will not be described in detail. Forexample, input signal receiver 834, input stream processor 836, storagedevice 837, decoders 838 and 844 and sample rate converter 856 performsubstantially as described above to provide the 3D half resolutionoutput 842, the 2D full resolution output 848 and 3D full resolutionoutput 858 to the selector 850.

Receiving device 800 further includes an encoder 860 coupled to theoutput of selector 850. Encoder 860 is also coupled to memory 862.Controller 854 is also coupled to encoder 860 as well as memory 862. Auser interface 870 is coupled to controller 854.

In operation, when selector 850 outputs either a 3D half resolutionsignal 842 or 3D full resolution signal 858, encoder 860 receives thedecoded pair of left view and right view images and stores the images inthe coded image buffer 864, shown as portion of memory 862. The encoder860 sets the left view image to be the reference image, also known as aninstantaneous decode refresh (IDR) frame. Encoder 860 apportions thereference image to select 4 pixel by 4 pixel (4×4) motion blocks and useP-frame unidirectional prediction to generate a motion vector for each4×4 sub-block of the right view image. It is important to note that theuse of a 4 pixel by 4 pixel block is based on standard practice.However, other block sizes may be used with corresponding level ofdifferences in results. The motion vector for each 4×4 block is used toencode the right view image which is stored in the coded image buffer864. Since the desired output of the encode process is the motionvectors rather than the actual coded image the motion vectors areextracted from the coded image signal and stored in a motion vectorbuffer 866, also shown as part of memory 862. The information stored incoded image buffer 864 following creation and storage of the motionvectors in motion vector buffer 866. As necessary, encoder 860 stepsthrough the remaining portions of the image signal to encode the nextimage pair until all images pairs are processed.

It is to be appreciated that coded image buffer 864 and motion buffer866 may reside on a single, shared memory device 862 or may be separateindividual buffers. In the embodiment shown in FIG. 8 with a singleshared memory device 862, an arbiter 863 is included. Arbiter 863arbitrates, or manages, access to the shared memory elements of memory862. Although FIG. 8 only shows two clients (i.e., encoder 860 andcontroller 854) accessing memory device 862, in most practical systemsthere are several independent clients all trying to access the samememory device, such as memory device 862. In other embodiments, memoryaccess may be individually controlled or controlled by a controllersimilar to control 854. It is the function of the arbiter 863 to makesure each client gets access at an assigned level of priority and withsufficient bandwidth and low enough latency to accomplish the taskassigned to each of the functional units.

The signal stored in motion vector buffer 866 is filtered to remove anyvertical component of any generated motion vector. The filtered resultis used as an indication, with picture level granularity of four pixelby four pixel image regions, of the horizontal disparity between theleft view and right view pictures. This filtered result, represented asa numerical entry, or as some other quantifiable difference indicator,is stored in disparity buffer 868, also shown as part of memory device862. It is to be appreciated that although the above functions areillustrated and described as being performed by encoder 860, one or moreof the functions could also be implemented by controller 854 or by fixedfunction hardware or a combination of hardware and software.

The array of horizontal disparity values stored in disparity buffer 868may be further filtered (e.g., spatially) by taking into account thedifferences in value between horizontally adjacent disparity values. Forexample, consider a 4×4 picture block B with a 4×4 picture block A toits immediate left and a 4×4 picture block C to its immediate right. Ifthe disparity between left and right picture views at block A is 6, andif the disparity between left and right picture views at block B is 4,and if the disparity between left and right picture views at block C is2, then a gradual shift is indicated in the disparity as a function ofthe progression from the left spatial position in the image pair to theright spatial position in the image pair. Assuming that this shift issmooth, the controller 854 can interpolate across 4×4 picture block Bassigning the left-most pixels with a disparity between 4 and 6 and theright most pixels with a disparity between 4 and 2. Although thisexample only looks at, or incorporated and processes, one 4×4 region tothe left and right, other filters could be employed that incorporatefurther regions in either direction. In addition, the same filterprinciples may be applied in the vertical direction to create andprocess a disparity map with pixel level granularity.

As needed, encoder 860, or alternatively the controller 854, generatesan occlusion map. Sharp changes in disparity value between horizontallyor vertically adjacent 4×4 blocks provide an indication of the existenceof edges between near field objects and far field objects in the stereoimage pair. On these edges, it is typically the case that the left eyeis able to see part of the near field object that is blocked from viewfor the right eye; and likewise the right eye will have a view of partof the object which is blocked for the left eye. These areas of visualmutual exclusivity of view between the left and right eyes createocclusions which can be detected while processing the disparity vectorsand signaled to a later image processing step. In later image processingsteps, it may be advantageous to apply a different processing of theseoccluded regions versus the processing applied to the overall picture.Together, the pixel level disparity map described above and theocclusion map form a dense disparity map. It is to be appreciated thatthere are several techniques known in the art to generate a densedisparity map from the pixel level disparity map and the occlusion map.

A dense disparity map of a stereo video signal reveals the convergencedepth of every pixel pair within the stereo image pair. When thisinformation is known and identified, the receiving device 800 is able todetermine how and where in the picture graphics overlays can bepresented in order to avoid unpleasant depth conflicts between video andgraphics. In addition, the dense disparity map can be used in order toallow receiving device 800 to modify the perceived 3D depth of a stereoimage by applying a disparity and occlusion aware scaling algorithm toone or both pictures of the stereo pair. This enables receiving device800 to implement features such as user controlled 3D depth.

Depth scaling using disparity and occlusion aware scaling permits easymodification of the range of perceived depth in the stereo image pair.For example, the range of depth in an image may be identified as theperceived distance between the furthest object behind the plane of theimage rendering surface and the object furthest in front of the plane ofthe image rendering surface. This depth range represents a visualstereoscopic ‘volume’. In one embodiment, the user may control thisvisual stereoscopic ‘volume’ with a simple user control, such as anup/down control via user interface 870. The control may be used toincrease or decrease the perceived range of depth in the stereo image.User interface 870 may be any known user interface including, but notlimited to, a remote control device, a plurality of buttons disposed onthe receiving device 804, a user preference screen generated by thereceiving device 804 and displayed on a display device, etc. Therefore,by generating a dense disparity map in the receiving device 804, usercontrolled depth scaling can be achieved.

Turning to FIG. 9, a flowchart of an exemplary process 900 forgenerating the dense disparity map is shown. The steps of process 900may be performed in receiving device 800. The steps of process 900 maysimilarly be performed in other receivers or set top box devices, suchas receiver 204 described in FIG. 2 or receiving device 504 described inFIG. 5. At step 902, the decoded pair of left view and right viewimages, as part of a signal, are received. The decoded pair of left andright view images are provided by a selector and may be a halfresolution 3D image signal or a full resolution 3D image signal.Further, the full resolution 3D image signal may include a sample rateconverted portion of the half resolution 3D image signal as describedabove. Step 902 may also include storing the images in memory, such asthe coded image buffer 864.

At step 904, the left view image is set to be the reference image orIDR. Although either image may be set as the reference image, it isadvantageous to use the left view image because, as described above, theleft view image is a full resolution image, thus reducing or eliminatingthe presence of image artifacts in the reference image. Next, at step906, the size and apportioning of motion blocks is selected. Asdescribed above, the motion block size may be 4×4 pixels, however, otherblock sizes are possible. At step 908, motion vectors are generatedusing P-frame unidirectional prediction for each of the correspondingsub-blocks of the right view image with reference to the left viewimage. Next, at step 910, the generated motion vectors for thesub-blocks are used to encode the right view image. At step 912, thefinal set of motion vectors from steps 908 and 910 is stored in amemory, such as motion vector buffer 866. In process 900 as described inthis manner, the left and right view images are processed as if theyexist temporally rather than spatially. In other words, although theleft and right images are intended to be viewed simultaneously in time,the motion processing is performed as if the left and right view imagesoccurred consecutively in time.

At step 914, the contents of the memory storing the encoded images(e.g., coded image buffer 864) are discarded, or erased, since thedesired output of the encode process steps described above is the motionvectors rather than the actual coded image the motion vectors. From step914, process 900 returns to step 904 to encode any remaining image pairsor portions until all images pairs are encoded and motion vectorsprocessed.

Next, at step 916, a granular disparity map is generated and stored byfiltering sets of motion vectors. The granular disparity map is anindication, with picture level granularity of equal to sub-block sizeselected at step 906, of the horizontal disparity between the left viewand right view pictures. In one embodiment, the motion vectors for a setof three consecutively located motion sub-blocks are compared andquantified to determine adjacent disparity values. The array ofhorizontal disparity values may then be further filtered, orinterpolated, at step 920, by taking into account the differences invalue between horizontally adjacent disparity values and furtherinterpolated vertically in a manner similar to that described above todetermine a pixel level disparity map.

Next, at step 922, an occlusion map is generated. As discussed above,any changes in disparity value between horizontally or verticallyadjacent blocks may be an indication of edges between near field and farfield objects in the stereo image pair. The occlusion map may begenerated at step 922 using either the original motion vectors stored atstep 912, or the disparity maps generated at steps 916 and/or 920. Inthe latter, the occlusion map may be generated from the motion vectorsby filtering or interpolating the motion vectors in a manner toreinforce the presence of edges. Finally, at step 924 the pixel leveldisparity map generated at step 920 and the occlusion map generated atstep 922 are combined to form a dense disparity map. It is to beappreciated that there are several techniques known in the art togenerate a dense disparity map from the pixel level disparity map andthe occlusion map. As described above, the dense disparity may be usedto permit such features as a user adjustable depth range in the 3Dimage.

The embodiments described in FIGS. 8 and 9 relate to generating a densedisparity map in a receiving device. The embodiments include receiving asignal that includes a desired bit stream. The desired bit stream mayfurther include, and be separated into one or more bit streamsrepresenting a 2D full resolution image signal as a single left eye orright eye image and a 3D partial or reduced (e.g., half) resolutionimage signal containing a reduced resolution left eye image and areduced resolution right eye image. The bit streams are decoded in orderto produce a signal having a left eye image and a right eye image,either at full or reduced resolution. The embodiments further describeencoding the left eye image as a reference image, predictively codingthe right eye image using the coded left eye image as the referenceimage, capturing motion indicators generated during encoding of theright eye image, and generating a dense disparity map between the lefteye image and right eye image using the motion indicators.

Although embodiments which incorporate the teachings of the presentdisclosure have been shown and described in detail herein, those skilledin the art can readily devise many other varied embodiments that stillincorporate these teachings. Having described preferred embodiments ofsystems and low bandwidth methods for encoding and broadcasting fullresolution 2D video, full resolution 3D video and half resolution 3Dvideo (which are intended to be illustrative and not limiting), it isnoted that modifications and variations can be made by persons skilledin the art in light of the above teachings. It is therefore to beunderstood that changes may be made in the particular embodiments of thedisclosure disclosed which are within the scope of the disclosure asoutlined by the appended claims.

What is claimed is:
 1. A method comprising: receiving a signalcomprising a three dimensional image signal representing a left eyeimage portion and a right eye image portion; encoding a frame in theleft eye image portion as a reference image using a two dimensionalencoding process; apportioning motion blocks in the left eye imageportion and right eye image portion, the motion blocks being a fixednumber of pixels; predictively coding, in the two dimensional encodingprocess, a frame in the right eye image portion using the frame in thecoded left eye image portion as the reference image, the predictivecoding using unidirectional prediction between the frame in the righteye image portion and the frame in the left eye image portion as if theframe in the right eye image portion and the frame in the left eyeportion occurred consecutively even though the frame in the right eyeimage portion and the frame in the left eye image are intended to beviewed simultaneously; capturing motion indicators generated from thetwo dimensional encoding process during the predictive coding of theframe in the right eye image portion; and generating a granulardisparity map between the frame in the left eye image portion and theframe in the right eye image portion by filtering the motion indicators,the granular disparity map including and indication of disparity withpicture level granularity equal to a size of the apportioned motionblocks; interpolating the granular disparity map to a pixel leveldisparity map by determining horizontally adjacent disparity values inat least three consecutively located motion blocks; and generating adense disparity map by combining the pixel level disparity map and anocclusion map, the occlusion map generated from the pixel disparity map.2. The method of claim 1, wherein the generating a granular disparitymap includes filtering motion indicators to remove vertical components.3. The method of claim 1, wherein the signal is an encoded signal, theencoded signal is encoded to include at least one of a full resolutiontwo-dimensional image, a reduced resolution three-dimensionalstereoscopic image, and a full resolution three-dimensional stereoscopicimage.
 4. The method of claim 3, wherein the encoded signal is encodedusing a three-dimensional encoding process.
 5. The method of claim 4,further comprising decoding the left eye image portion and the right eyeimage portion in the encoded signal.
 6. The method of claim 1, whereinmethod is performed in a set-top box.
 7. The method of claim 1, whereinthe two dimensional encoding process is an H.264 encoding process. 8.The method of claim 1, wherein the motion indicators include at leastone motion vector.
 9. The method of claim 1, wherein the fixed number ofpixels in each apportioned motion block is a 4×4 block of pixels. 10.The method of claim 9, wherein the interpolating includes processing themotion indicators to generate motion indications for individual pixels.11. An apparatus comprising: a receiver that receives a signalcomprising a three dimensional image signal representing a left eyeimage portion and a right eye image portion; an encoder, coupled to thereceiver, that encodes a frame in the left eye image portion as areference image using a two dimensional encoding process, the encoderpredictively codes, using the two dimensional encoding process, a framein the right eye image portion using the frame in the coded left eyeimage portion as the reference image, the predictive coding usingunidirectional prediction between the frame in the right eye imageportion and the frame in the left eye image portion as if the frame inthe right eye image portion and the frame in the left eye image portionoccurred consecutively even though the frame in the right eye imageportion and the frame in the left eye image portion are intended to beviewed simultaneously and the encoder captures motion indicatorsgenerated from the two dimensional encoding process during thepredictive coding of the frame in the right eye image portion; and acontroller, coupled to the encoder, that apportions motion blocks in theleft eye image portion and right eye image portion, the motion blocksbeing a fixed number of pixels and generates a granular disparity mapbetween the frame in the left eye image portion and the frame in theright eye image portion by filtering the motion indicators, the granulardisparity map including and indication of disparity with picture levelgranularity equal to a size of the apportioned motion blocks, thecontroller further interpolates the granular disparity map to a pixellevel disparity map by determining horizontally adjacent disparityvalues in at least three consecutively located motion blocks andgenerates a dense disparity map by combining the pixel level disparitymap and an occlusion map, the occlusion map generated from the pixellevel disparity map.
 12. The apparatus of claim 11, wherein thecontroller generates the granular disparity map by filtering the motionindicators to remove vertical components.
 13. The apparatus of claim 11,wherein the signal is an encoded signal, the encoded signal is encodedto include at least one of a full resolution two-dimensional image, areduced resolution three-dimensional stereoscopic image, and a fullresolution three-dimensional stereoscopic image.
 14. The apparatus ofclaim 13, wherein the encoded signal is encoded using a threedimensional encoding process.
 15. The apparatus of claim 14, furthercomprising a decoder that decodes the left eye image portion and theright eye image portion in the encoded signal.
 16. The apparatus ofclaim 15, wherein the two dimensional encoding process is an H. 264encoding process.
 17. The apparatus of claim 11, wherein the motionindicators include at least one motion vector.
 18. The apparatus ofclaim 11, wherein the fixed number of pixels in each apportioned motionis a 4×4 block of pixels.
 19. The apparatus of claim 18, wherein thecontroller interpolates motion indications for individual pixels.
 20. Anapparatus comprising: means for receiving a signal comprising a threedimensional signal representing a left eye image portion and a right eyeimage portion; means for encoding a frame in the left eye image portionas a reference image using a two dimensional encoding process; means forapportioning motion blocks in the left eye image portion and right eyeimage portion, the motion blocks being a fixed number of pixels; meansfor predictively coding, in the two dimensional encoding process, aframe in the right eye image portion using the frame in the coded lefteye image portion as the reference image, the means for predictivecoding using unidirectional prediction between the frame in the righteye image portion and the frame in the left eye image portion as if theframe in the right eye image portion and the frame in the left eyeportion occurred consecutively even though the frame in the right eyeimage portion and the frame in the left eye image portion are intendedto be viewed simultaneously; means for capturing motion indicatorsgenerated from the two dimensional encoding process during thepredictive coding of the frame in the right eye image portion; and meansfor generating a granular disparity map between the frame in the lefteye image portion and the frame in the right eye image portion byfiltering the motion indicators, the granular disparity map including anindication of disparity with picture level granularity equal to a sizeof the apportioned motion blocks, means for interpolating the granulardisparity map to a pixel level disparity map by determining horizontallyadjacent disparity values in at least three consecutively located motionblocks; and means for generating a dense disparity map by combining thepixel level disparity map and an occlusion map, the occlusion mapgenerated from the pixel level disparity map.
 21. The apparatus of claim20, wherein the generating a granular disparity map includes filteringmotion indicators to remove vertical components.
 22. The method of claim20, wherein the signal is an encoded signal, the encoded signal isencoded to include at least one of a full resolution two-dimensionalimage, a reduced resolution three-dimensional stereoscopic image, and afull resolution three-dimensional stereoscopic image.
 23. The method ofclaim 22, wherein the encoded signal is encoded using athree-dimensional encoding process.
 24. The method of claim 23, furthercomprising decoding the left eye image portion and the right eye imageportion in the encoded signal.
 25. The apparatus of claim 20, whereinthe two dimensional encoding process is an H.264 encoding process. 26.The apparatus of claim 20, wherein the motion indicators include atleast one motion vector.
 27. The apparatus of claim 20, wherein thefixed number of pixels in each motion block is a 4×4 block of pixels.28. The apparatus of claim 27, wherein the means for interpolatingincludes means for processing the motion indicators to generate motionindications for individual pixels.